Efficient multiview depth video coding using depth synthesis prediction

The view synthesis prediction (VSP) method utilizes interview correlations between views by generating an additional reference frame in the multiview video coding. This paper describes a multiview depth video coding scheme that incorporates depth view synthesis and additional prediction modes. In the proposed scheme, we exploit the reconstructed neighboring depth frame to generate an additional reference depth image for the current viewpoint to be coded using the depth image-based-rendering technique. In order to generate high-quality reference depth images, we used pre-processing on depth, depth image warping, and two types of hole filling methods depending on the number of available reference views. After synthesizing the additional depth image, we encode the depth video using the proposed additional prediction modes named VSP modes; those additional modes refer to the synthesized depth image. In particular, the VSP_SKIP mode refers to the co-located block of the synthesized frame without the coding motion vectors and residual data, which gives most of the coding gains. Experimental results demonstrate that the proposed depth view synthesis method provides high-quality depth images for the current view and the proposed VSP modes provide high coding gains, especially on the anchor frames. C 1 Introduction The three-dimensional(3D) video provides depth impression of the observed scenery with slight different viewpoints between the left and right eyes, which stimulates the human brain to perceive distance of objects. Due to mature 3D technologies from capturing to display, awareness and interests are rapidly increasing among users. 1–3 A key issue in 3D video technology is how to produce a comfortable 3D scene minimizing visual fatigues. Since most of the visual fatigues are induced by the improper camera baseline, it can be solved by selecting two proper viewpoint images among various viewpoint images. In such application, a sufficient number of viewpoint images should be sent to the 3D displays. However , the huge amount of data due to the multiple views is a serious problem for service; hence, we need to develop an efficient video coding. In response to such needs and interests, many researchers developed various data formats and coding methods for rendering a 3D scene. 4 Particularly, moving picture experts group (MPEG) and joint video team have developed the multiview video coding (MVC), which compresses multi-view videos using high correlations between views. 5 It is the latest coding standard designed for coding the multiview videos efficiently. It employs an interview/temporal prediction structure based on …